refactor(memory): Improve Chat Memory logic and efficiency #4065
Conversation
Refactors `MessageWindowChatMemory` to improve performance and extensibility by abstracting the storage layer. Introduces a `ChatMemoryRepository` interface to decouple memory logic from the underlying storage. This allows for future persistent implementations (e.g., Redis, JDBC). The default `InMemoryChatMemoryRepository` now uses an efficient `refresh` method to apply only deltas, reducing overhead in long conversations. Signed-off-by: Juntar Park <[email protected]> Signed-off-by: potato <[email protected]>
juntae6942
force-pushed
the
fix/chat-memory-efficiency-3945
branch
from
6becb80 to
6bf4f28
Compare
|
This might need some discussion. Some NoSQL databases (like MongoDB?) only support full message overwrite and do not support incremental updates or deletions. And after |
|
@YunKuiLu Thank you for the insightful feedback! Both points you raised are important, and I had them in mind as well. Regarding the concern for NoSQL databases that only support full overwrites, that is a very valid point. My intention with this PR was to first establish the foundational, more efficient repository pattern with the new interface and the default in-memory implementation. For specific database implementations that don't support incremental updates, we can ensure they rely on a saveAll-style full overwrite to maintain compatibility, while other databases can leverage the benefits of the new refresh method. I was also fully aware of the need to update the other ChatMemoryRepository implementations (jdbc, cassandra, neo4j). My plan was to first get this core refactoring of the interface and the base implementation merged. I believed that separating the changes this way would keep the pull requests smaller and more focused, making them easier to review and discuss. If this approach sounds reasonable to you, I plan to submit follow-up PRs for the other modules once this foundational PR is merged. Thank you again for the thoughtful review. |
Title:
Refactor(memory): Improve logic and efficiency of Chat Memory
Body:
Closes: #3945
This pull request addresses the performance overhead in MessageWindowChatMemory by refactoring its internal logic and abstracting the storage layer. The previous implementation replaced the entire message list on every update, which is inefficient for long-running conversations.
Key Changes
Introduced ChatMemoryRepository: A new interface is introduced to abstract the storage mechanism, decoupling the chat memory logic from the in-memory implementation. This design allows for future persistent implementations (e.g., Redis, JDBC).
Efficient InMemoryChatMemoryRepository: The default InMemoryChatMemoryRepository now uses an efficient refresh method to apply only deltas (additions/deletions) instead of replacing the entire list, significantly reducing overhead.
Updated MessageWindowChatMemory: The class is refactored to use the new ChatMemoryRepository, delegating all state mutations to the repository layer.
Updated Tests: Comprehensive unit tests have been added and updated to ensure the reliability and correctness of the new implementation.